Papers with Llama-3-8B model
SYNTHVERIFY: Enhancing Zero-Shot Claim Verification through Step-by-Step Synthetic Data Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for claim verification are inefficient or rely on external documents. |
| Approach: | They propose a step-by-step prompting-based synthetic data generation framework to enhance zero-shot claim verification. |
| Outcome: | The proposed framework bridges LLMs’ knowledge gaps in specialized domains without access to external corpora or sacrificing generalizability. |
Efficient Domain Continual pretraining by Mitigating the Stability Gap (2025.acl-long)
Copied to clipboard
| Challenge: | Continual pretraining is an important approach for Large Language Models to improve their performance in target domains, learn new topics and languages, and even boost their general capabilities. |
| Approach: | They propose a training strategy that mitigates instability by increasing the number of epochs, along with two data sampling strategies targeting data domain relevance and corpus distribution. |
| Outcome: | The proposed training strategy improves the average medical task performance of the OpenLlama-3B model from 36.2% to 40.7% using only 40% of the original training budget, while also enhancing general task performance without causing forgetting. |